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Convergence in Probability 


7.2.5 Convergence in Probability 

Convergence in probability is stronger than convergence in distribution. In particular, for a sequence X\, X 2 , X 3 , ■ • • to 
converge to a random variable X , we must have that P(\X n — X\ > e) goes to 0 as TL — y OO, for any e > 0. To say that 
X n converges in probability to X, we write 


X n —> X. 

Here is the formal definition of convergence in probability: 

Convergence in Probability 


A sequence of random variables X\, X 2 , X 3 , • ■ ■ converges in probability to a random variable X, shown by X n —y X, 
if 


lim P( \X n — X\ > e) = 0, for all e > 0. 


Example 7.8 

P 

Let X n ~ Exponential (n) , show that X n —y 0. That is, the sequence X\, X 2 , X 3 , ■ • • converges in probability to 
the zero random variable X . 

• Solution 

o We have 

lim P(X n > e) ( since X n > 0 ) 

n-¥oo y ' 

lim e~ ne ( since X n ~ Exponential (n ) ) 

n—>00 

0, for all e > 0. 



Example 7.9 

Let X be a random variable, and X n = X + Y n , where 

1 (J 2 

EY n = —, Var Y n = —, 
n n 

p 

where G > 0 is a constant. Show that X n —y X. 

• Solution 

o First note that by the triangle inequality, for all a, b E R, we have |o + 6| < | a | + 1 6 1 . Choosing 
a = Y n — EY n and b = EY n , we obtain 


Y n I < 1 Y n -EY n \ + ~. 


n 


Now, for any e > 0, we have 
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< 


P(\Y n \ > 

e ) 

p(\Y n - 

EY n \ + -> e 

V 

n 

p(\Y n - 

EY n 1 > e- - 

V 

n 

Var(y n ) 



(*-*r 

a 2 


i \ 2 


as n —> oo. 


Therefore, we conclude X^ 


P 


X. 


(by Chebyshev’s inequality) 


p 

As we mentioned previously, convergence in probability is stronger than convergence in distribution. That is, if X n — > X, 
d 

then X n — > X. The converse is not necessarily true. For example, let X\, X 2 , X 3 , • • • he a sequence of i.i.d. 

Bernoulli (^-) random variables. Let also X ~ Bernoulli (^-) be independent from the Xi's. Then, X n — > X. 
However, X n does not converge in probability to X, since \X n — X\ is in fact also a Bernoulli (-^-) random variable and 

P( \X n — X\ > e) = —, for 0 < e < 1. 

2 

d 

A special case in which the converse is true is when X n —> C, where C is a constant. In this case, convergence in distribution 
implies convergence in probability. We can state the following theorem: 


d p 

Theorem 7.2 If X n —> C, where C is a constant, then X n —> C. 

• Proof 

d 

o Since X n — > C, we conclude that for any e > 0, we have 

lim F Xn (c — e) 

n—>00 

lim F Xn (c+ 

n—>00 Z 


We can write for any e > 0, 


0 , 

1 . 
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lim P( \X n — c| > e) = lim 


P{X n < c-e)+P(X n > c + e) 


= lim P(X n < c - e) + lim P(X n >c + e) 

n—>oo n—>oo 

= lim Fx n (c — e) + lim P(X n > c + e) 

n—>oo n n—>oo 

= 0 + lim P{X n >c + e) 


< lim P(X n > c + -) 

n—>oo Z 

= 1- lim F Xn (c +|) 

n—>oo Z 

= o 


Since lim P(\X n — cl > e) > 0, we conclude that 

n—¥oo v ' 

lim P(\X n - c\ > e) = 0, 


(since lim F x „ (c - e) = 0) 


(since lim F x „(c +-) = 1). 

n—too Z 


for all e > 0, 


which means Xn 


C. 


The most famous example of convergence in probability is the weak law of large numbers (WLLN). We proved WLLN in 
Section 7.1.1 . The WLLN states that if X\, X%, X;> t , ■ ■ ■ are i.i.d. random variables with mean EXj = fl < OO, then the 
average sequence defined by 


^ _ X\ + X2+.. -+X n 

-™-n — 

n 

converges in probability to fl. It is called the "weak" law because it refers to convergence in probability. There is another version 
of the law of large numbers that is called the strong law of large numbers (SLLN ). We will discuss SLLN in Section 7.2.7 . 
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